“Big data” is a broad term for datasets so large or complex that traditional data processing applications are often inadequate. For example, a social networking system can run several application services that pertain to big data. The term “big data” also often refers to the use of predictive analytics or other methods to extract values from data. For example, analysis of datasets can find new correlations, trends, patterns, categories, etc. Such analyses rely on machine learning and often consumes a large amount of computational resources (e.g., memory capacity, processor capacity, and/or network bandwidth).
A typical machine learning workflow may include building a model from a sample dataset (referred to as a “training set”), evaluating the model against one or more additional sample datasets (referred to as a “validation set” and/or a “test set”) to decide whether to keep the model and to benchmark how good the model is, and using the model in “production” to make predictions or decisions against live input data captured by an application service. The training set, the validation set, and/or the test set can respectively include pairs of input datasets and expected output datasets that correspond to the respective input datasets.
Various web-based or mobile applications often rely on machine learning models to process large and complex “big data” to provide application services (e.g., personalized or targeted application services) to a large number of users. There is frequently a need for higher accuracy and/or consistency models while the requirements of these models are ever evolving. Experiments involving the training and evaluation of these models nevertheless take time and are typically the manual burdens of one or more developers or analysts.
The figures depict various embodiments of this disclosure for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of embodiments described herein.